Systems and methods for detecting corrupt or inaccurate sensory representations

ABSTRACT

A system for monitoring neural activity of a living subject is provided. The system may comprise a correspondence module configured to be in communication with (1) a neural module and (2) one or more additional modules comprising a sensing module, another neural module, and/or a data storage module. The neural module(s) are configured to collect neural data indicative of perceptions experienced by the living subject. The sensing module may be configured to collect (1) sensor data indicative of real-world information about an environment around the living subject, and/or (2) sensor data indicative of a physical state or physiological state of the living subject. The data storage module may be configured to store prior neural data and/or prior sensor data. The correspondence module may be configured to measure a correspondence (a) between the neural data collected by the neural module(s) and the sensor data collected by the sensing module, (b) between the neural data collected by two or more neural modules, and/or (c) between the neural data collected by the neural module(s) and the prior data stored in data storage module. The measured correspondence can be used to determine a presence, absence, or extent of a potential cognitive or physiological disturbance of the living subject.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation application of International PatentApplication No. PCT/US2017/052589, filed on Sep. 20, 2017, whichapplication claims priority to U.S. Provisional Patent Application No.62/397,276, filed on Sep. 20, 2016, which applications are incorporatedherein by reference in their entirety.

BACKGROUND

Disorders of the nervous system, such as traumatic brain or nerve injuryor neuropsychiatric illness can result in a dramatic misalignmentbetween a person's perception of their surroundings and the true stateof their physical environment. Such a misalignment may be indicative ofcorrupt or inaccurate sensory perceptions. This is particularly true inthe case of sensory hallucinations. Patients suffering from mentaldisorders such as schizophrenia may experience a sensory stimulus thatdoes not coincide with the patients' real-world environment. Forexample, a patient may experience an auditory perception (appear to hearsounds such as voices), when in fact no such sounds are being emanatedin the surroundings. In some cases, the patient may be aware that thesensory stimulus is not real, and is able to revert back to reality.However, in other cases, the patient may be unable to distinguishhallucinated perceptions from reality. In still other cases, the patientmay have partial awareness that the sensory stimulus is not real, andmay be able to partially revert back to reality.

Thus, there is a need for systems and methods that can detectinaccuracies or distortions in neural representations of sensorystimuli, with respect to real-world environmental stimuli. Suchinaccuracies or distortions may indicate that the neural representationsare corrupted, and that a patient may be experiencing hallucinations orother corrupt or inaccurate sensory perceptions. There is a need tointervene when these events occur, potential alerting the patient, aprovider, a caretaker, or a monitoring system. There is also a furtherneed for systems and methods that can correct for these inaccurateneural representations, by reducing or eliminating the corrupt orinaccurate perceptions.

SUMMARY

The system and methods described herein can address at least the aboveneeds. Embodiments of the system may be particularly well suited for usein brain research applications, as well as in clinical applications,such as in the development of methods for treating nervous systemdisorders.

According to one aspect of the invention, a system for monitoring neuralactivity is provided. The system may comprise: a neural data analysismodule configured to extract neural data from a plurality of neuralsignals that are collected using one or more neural interface probesimplanted into a living subject; a sensing module configured to collectsensor data indicative of real-world environmental stimuli in a vicinityof the living subject; and a correspondence module in communication withthe neural data analysis module and the sensing module, wherein thecorrespondence module is configured to generate an output based on theneural data and the sensor data, wherein the output is indicative of anagreement or disagreement between external sensors and neural databetween the real-world environmental stimuli and neural perceptionsexperienced by the living subject.

In some embodiments, the neural data analysis module may be configuredto extract neural data from a plurality of signals that are collectedfrom the nervous system of a living subject using one or more devices.The sensing module may be configured to collect sensor data indicativeof real-world signals from an environment around or within the livingsubject. The correspondence module may be in communication with theneural data analysis module and the sensing module, and configured togenerate an output based on the neural data and the sensor data, whereinthe output is indicative of a correspondence between the environment andneural perceptions experienced by the living subject.

In some embodiments, the correspondence module may be configured todetect corrupt or inaccurate sensory representations from thecorrespondence based on statistical models, information theory, ormachine learning algorithms.

In some embodiments, the plurality of signals collected from the nervoussystem may comprise electrical signals, magnetic signals, and/or opticalsignals. The aforementioned one or more devices may comprise a neuralinterface probe provided in a massively parallel configuration. Forexample, the neural interface probe may comprise a microwire bundlebonded onto a CMOS sensing array. In other embodiments, the neuralinterface probe may comprise an array of silicon microelectrode probesthat are bonded onto a CMOS sensing array such that each electrode siteis routed to a unique array position. In some cases, the one or moredevices may comprise an electroencephalography (EEG) device. In otherembodiments, there may be as few as one sensor detecting activity in thenervous system.

In some embodiments, the neural data may be represented as one or moreanalog or digital signals representing activity recorded in the nervoussystem. In some cases, the neural data may be stored from previouslyrecorded activity in the nervous system of the living subject or otherliving subjects.

In some embodiments, the sensing module may comprise a plurality ofsensors selected from the group consisting of vision sensors, audiosensors, touch sensors, location sensors, inertial sensors, proximitysensors, heart rate monitors, temperature sensors, altitude sensors,attitude sensors, pressure sensors, humidity sensors, vibration sensors,chemical sensors, and electromagnetic field sensors. One or more sensorsof the sensing module may also record data from a region of the nervoussystem. For instance, the sensors of the sensing module may record datafrom a different region of the brain than the region from which theneural interface probe records. In some cases, the sensors may recorddata from the same region of the brain from which the neural interfaceprobe records. One or more of the plurality of sensors in the sensingmodule may be provided in a mobile device, or in a wearable deviceconfigured to be worn by the living subject.

In some embodiments, the one or more neural interface probes may beimplanted in different areas of the brain of the living subject that areassociated with different sensory processing. The different sensoryprocessing may include visual, auditory, tactile, taste, smell,position/movement, and/or interoception processing. One or more sensorsmay be configured to collect real-world environmental stimuli related toeach of the different sensory processing. In one instance, at least oneneural interface probe may be implanted in an area of the livingsubject's brain associated with auditory processing, and wherein thesensing module may comprise at least one microphone configured tocollect audio data in the vicinity of the living subject. In anotherinstance, at least one neural interface probe may be implanted in anarea of the living subject's brain associated with visual processing,and wherein the sensing module may comprise at least one cameraconfigured to collect image data of the living subject's surrounding. Inyet another instance, at least one neural interface probe may beimplanted in an area of the living subject's brain associated withspatial or location awareness, and wherein the sensing module maycomprise at least one global positioning sensor (GPS) sensor configuredto collect positional data of the living subject.

The correspondence module can be configured to determine correspondencebetween the neural data and the sensor data. Additionally or optionally,the correspondence module can be configured to determine correspondencebetween different sets of neural data. In some embodiments, thecorrespondence module can be implemented via a statistical model. Insome cases, the correspondence module can be configured to performinformation theoretic calculations. The correspondence module can beimplemented via machine learning algorithms.

In some embodiments, the correspondence module may be configured toanalyze the neural data and the sensor data using a statistical model,information theory, or machine learning. The statistical model may bebased on either frequentist or Bayesian statistics. In some instances,the information theoretic approach may be based on mutual information orrelated quantities. In some instances, the statistical model may beimplemented in a neural network or any other approach to machinelearning. The calculations in the correspondence module may becalculated using data from two or more modules of the same type, ormodules of different types. The inputs and outputs of this module may beunivariate or multivariate.

In some embodiments, the correspondence module may further comprise adecoder configured to reconstruct neural representations of sensoryperceptions from the neural data. In one instance, the decoder maycomprise a speech recognition software configured to reconstruct neuralrepresentations of speech and sounds, and wherein said neuralrepresentations may be recorded within the neural signals collected byone or more neural interface probes that are implanted in an area of theliving subject's brain associated with auditory processing. Thecorrespondence module may be configured to compare the reconstructedneural representations of speech and sounds to actual audio datarecorded by the sensing module in the living subject's vicinity, so asto determine (1) whether the reconstructed neural representations ofspeech and sounds correspond to real-life audio stimuli, or (2) whetherthe reconstructed neural representations of speech and sounds correspondto auditory hallucinations. In another instance, the decoder maycomprise an image recognition software configured to reconstruct neuralrepresentations of visual data, and wherein said neural representationsmay be recorded within the neural signals collected by one or moreneural interface probes that are implanted in an area of the livingsubject's brain associated with visual processing. The correspondencemodule may be configured to compare the reconstructed neuralrepresentations of visual data to actual image data recorded by thesensing module in the living subject's vicinity, so as to determine (1)whether the reconstructed neural representations of visual datacorrespond to real-life visual stimuli, or (2) whether the reconstructedneural representations of visual data correspond to visualhallucinations.

The correspondence module can be configured to generate an error signalbased on the degree of correspondence between neural data and sensordata. Additionally or optionally, the correspondence module canconfigured to generate an error signal based on different sets of neuraldata. A high error signal may result from low correspondence, and may beindicative that the living subject is experiencing inaccurate neuralperceptions. The high error signal may indicate that patterns of neuralactivity are atypical and/or unhealthy patterns of neural activity.Conversely, a low error signal may result from high correspondence, andmay indicate that the living subject is experiencing neural perceptionsmatching the real-world environmental stimuli. The low error signal mayindicate that patterns of neural activity are typical and/or healthypatterns of neural activity. In some cases, the high error signal may beindicative of the living subject experiencing a hallucination episode.

In some embodiments, the correspondence module may further drive anerror detector configured to generate an error signal related to thelack of correspondence. The error signal may be indicative that theliving subject is experiencing inaccurate neural perceptions when theperception error exceeds the predetermined correspondence threshold. Alow error signal may indicate that the living subject is experiencingneural perceptions matching the real-world environmental stimuli,whereas a high error signal may be indicative of the living subjectexperiencing an episode of inaccurate sensory perception. In some cases,the error signal may comprise one or more visual signals, audio signals,or vibration signals that are used to alert the living subject or ahealthcare provider entity. The system may be in communication with aremote server associated with a healthcare provider entity.

The error signal may be derived from a technique such as canonicalcorrelation analysis (CCA) or any other linear or non-linear analoguethat finds the correspondence between lower-dimensional representationsof the sensor data and the neural data. The mapping from such atechnique can generate a correspondence, and a lack of correspondencecan drive the error signal.

The error signal may be vector-valued. It may simultaneously containdistinct values corresponding to individual measures of correspondencein different combinations of sensors and neural data streams. It mayalso contain values corresponding to analysis of previous measurementsor values produced by any computation.

The error signal may in some instances be driven not by lack ofcorrespondence between neural and sensor data, but simply by neural datawhich is determined to be unlikely in a healthy mental state. Forexample, the neural data might exhibit patterns characteristic of ahallucination, based on previous episodes of health or hallucination inthe same or other subjects, even if the sensor data is insufficient orunnecessary for determining this.

The error signal may be processed by a decision module, which may usethe strength of the evidence of hallucination represented by the errorsignal to generate a decision about system action. The decision modulemay be based on a simple error signal threshold. Alternatively, thedecision module may be based on Bayesian inference about the errorsignal in combination with a prior model of the distribution of thatsignal under various scenarios. Alternatively, the decision module maybe based on machine learning techniques related to classification.

The decision module may comprise different algorithms and/or thresholdsfor different specific decisions. For example, the decision aboutwhether to alert the user may require a certain error signal magnitude,whereas the decision about whether to administer medication may requireanother error signal magnitude.

In some embodiments, the system may further comprise a neural stimulatorconfigured to receive the output from the decision module, and generateneural stimuli depending indirectly on the strength of the error signal.The neural stimulator may be configured to receive an action signal fromthe decision module, and generate neural stimuli depending on the actionsignal. The action signal may be related to the correspondence betweenthe neural data and the sensor data. The neural stimulator may beconfigured to generate the neural stimuli when the action signalindicates that the correspondence between the neural data and the sensordata is low. In some instances, the neural stimulator may be animplanted medical device configured to automatically dispense apredetermined dosage of medication to the living subject when the actionsignal indicates that the correspondence between the neural data and thesensor data is low. The neural stimulator may also be an implantedmedical device configured to automatically vary a dispensed dosage ofmedication to the living subject depending on the degree ofcorrespondence. The medication may be designed to alter a mental stateof the living subject to improve the expected correspondence between theneural data and the sensor data, or to rectify the expected neural data.In some embodiments, the neural stimulator may be provided on one ormore neural interface probes, and configured to deliver electricalpulses to the living subject when the correspondence is low. Theelectrical pulses can be designed to alter a mental state of the livingsubject to improve the correspondence.

In some embodiments, the neural stimulator may thus generate the neuralstimuli when the correspondence is below a threshold, or when itexhibits some other properties that trigger the decision module to issuea neural stimulation directive. In one instance, the neural stimulatormay be an implanted medical device configured to automatically dispensea predetermined dosage of medication to the living subject when thedecision module makes such an indication. Alternatively, the neuralstimulator may be an implanted medical device configured toautomatically vary a dispensed dosage of medication to the livingsubject in such a situation. The medication may be designed to alter amental state of the living subject to improve correspondence. In somecases, the neural stimulator may be provided on the one or more neuralinterface probes, and configured to deliver electrical pulses to theliving subject. The electrical signals may be designed to alter a mentalstate of the living subject to improve the degree of correspondence.

In some embodiments, the system may exhibit negative feedback such thatthe output of the decision module results in actions that improvecorrespondence, reducing the error signal, and causing the decisionmodule to cease the action. For instance, upon detection of an episodeof inaccurate sensory processing, the decision module may administer amedication that reduces or eliminates the episode. This may reduce theerror signal, resulting in a decrease in the dosage of the medicationadministered.

In some embodiments, the correspondence module may be configured toreceive and process (1) the neural data from the neural data analysismodule and (2) the sensor data from the sensing module, substantially inreal-time.

In some embodiments, the correspondence module may receive sensor dataor neural data that was previously recorded. This may occur via on-boardstorage of that data, or by sending it from another module. In someinstances, previously recorded sensor data or neural data may also beused to compute correspondence, in conjunction with data recordedsubstantially in real-time. This previously recorded data may be used toestablish a baseline or prior distribution of data for a sensory orneural modality.

In some embodiments, the correspondence module may contain labeled datathat algorithms can use to make a determination about correspondence.For example it may contain many examples of neural data patterns, someof which are labeled as “normal” and others of which are labeled as“atypical”. For example, a machine learning classifier could use theselabels to assist in making a classification of newly recorded data.

In some embodiments, labeled data may be added to the module after thedevice is implanted or attached, via wired or wireless means, or viaprocessing of signals that it directly records from the nervous system.

In some embodiments, the correspondence module may be modified accordingto the contents of the labeled data that it can access at any giventime. For example, a classifier may be trained on this labeled data toimprove a determination of correspondence based on new data.

In some embodiments, labels may be provided to data based on externalactions by authorized users. For example, the patient him/herself or aclinician might examine the data in real time or offline and label thedata as being associated with a normal state or with a hallucination,based on their subjective assessment or any other analysis.

According to another aspect of the invention, a system for monitoringneural activity of a living subject is provided. The system may comprisea correspondence module configured to be in communication with (1) aneural module and (2) one or more additional modules comprising asensing module, another neural module, and/or a data storage module. Theneural module(s) may be configured to collect neural data indicative ofperceptions experienced by the living subject. The sensing module may beconfigured to collect (1) sensor data indicative of real-worldinformation about an environment around the living subject, and/or (2)sensor data indicative of a physical state or physiological state of theliving subject. The data storage module may be configured to store priorneural data and/or prior sensor data. The correspondence module may beconfigured to measure a correspondence (a) between the neural datacollected by the neural module(s) and the sensor data collected by thesensing module, (b) between the neural data collected by two or moreneural modules, and/or (c) between the neural data collected by theneural module(s) and the prior data stored in data storage module. Themeasured correspondence can be used to determine a presence, absence, orextent of a potential cognitive or physiological disturbance of theliving subject.

In some embodiments, the measured correspondence may comprise a level ofagreement or disagreement (a) between the neural data collected by theneural module(s) and the sensor data collected by the sensing module,(b) between the neural data collected by two or more neural modules,and/or (c) between the neural data collected by the neural module(s) andthe prior data stored in data storage module. The level of agreement ordisagreement may be measured along a continuum. The level of agreementor disagreement may comprise a multivariate vector. The correspondencemodule may be configured to measure the correspondence using statisticalmodels, information theory, or machine learning algorithms. Thecorrespondence module may be configured to measure the correspondence(b) between the neural data collected by the two or more neural modules,without requiring or utilizing some or all of the sensor data collectedby the sensing module. The correspondence module may be configured tomeasure the correspondence (c) between the neural data collected by theneural module(s) and the prior data stored in data storage module,without requiring or utilizing some or all of the sensor data collectedby the sensing module.

In some embodiments, the system may further comprise a decision moduleconfigured to determine the presence, absence, or extent of thepotential cognitive or physiological disturbance based on the measuredcorrespondence. The decision module may be further configured togenerate one or more alerts to the living subject and/or a healthcareprovider. The alerts may be indicative of the presence, absence, orextent of the potential cognitive or physiological disturbance. In someembodiments, the decision module may be further configured to generateone or more control signals to one or more therapeutic delivery devices,to deliver electrical stimulation and/or pharmaceutical intervention tomitigate the potential cognitive or physiological disturbance.

A method for monitoring neural activity of a living subject is providedin accordance with a further aspect of the invention. The method maycomprise: providing a correspondence module configured to be incommunication with (1) a neural module and (2) one or more additionalmodules comprising a sensing module, another neural module, and/or adata storage module. The neural module(s) may be configured to collectneural data indicative of perceptions experienced by the living subject.The sensing module may be configured to collect (1) sensor dataindicative of real-world information about an environment around theliving subject, and/or (2) sensor data indicative of a physical state orphysiological state of the living subject. The data storage module maybe configured to store prior neural data and/or prior sensor data. Themethod may further comprise measuring, with aid of the correspondencemodule, a correspondence (a) between the neural data collected by theneural module(s) and the sensor data collected by the sensing module,(b) between the neural data collected by two or more neural modules,and/or (c) between the neural data collected by the neural module(s) andthe prior data stored in data storage module. The measuredcorrespondence can be used to determine a presence, absence, or extentof a potential cognitive or physiological disturbance of the livingsubject.

A method for monitoring neural activity is provided in accordance withanother aspect of the invention. The method may comprise: obtainingneural data from a neural data analysis module, wherein the neural datais extracted from a plurality of neural signals collected using one ormore neural interface probes implanted into a living subject; obtainingsensor data from a sensing module, wherein the sensor data is indicativeof real-world environmental stimuli in a vicinity of the living subject;and generating, via a correspondence module in communication with theneural data analysis module and the sensing module, an output based onthe neural data and the sensor data, wherein the output is indicative ofcorrespondence between the real-world environmental stimuli and neuralperceptions experienced by the living subject.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present invention will be obtained by reference to thefollowing detailed description that sets forth illustrative embodiments,in which the principles of the invention are utilized, and theaccompanying drawings (also “Figure” and “FIG.” herein), of which:

FIG. 1 illustrates a neural data correspondence system, in accordancewith some embodiments;

FIG. 2 illustrates a closed-loop neural data correspondence systemcomprising a neural stimulator, in accordance with some embodiments;

FIG. 3 illustrates the communication between different body-mountedsensors, an implanted neural interface probe, a neural data analysismodule, and a correspondence module, in accordance with someembodiments;

FIG. 4 illustrates the reconstruction of perceptions from neural dataand correspondence of the perceptions with real-world sensor data, inaccordance with some embodiments;

FIG. 5 is a flowchart of a method of monitoring and correcting neuralperception errors, in accordance with some embodiments; and

FIG. 6 illustrates a system for monitoring neural activity of a livingsubject, in accordance with some embodiments.

DETAILED DESCRIPTION

While various embodiments of the invention have been shown and describedherein, it will be obvious to those skilled in the art that suchembodiments are provided by way of example only. Numerous variations,changes, and substitutions may occur to those skilled in the art withoutdeparting from the invention. It should be understood that variousalternatives to the embodiments of the invention described herein may beemployed.

Embodiments of the disclosure described herein can enable real-timedetection and correction of inaccurate neural perception applicable tocertain healthcare areas. For example, the methods and systems disclosedherein can be used to monitor the mental states of patients sufferingfrom certain mental disorders, and/or improve their state by correctingfor such inaccurate neural perception errors. The data that is collectedcan be used to help healthcare providers and users effectively managethose mental disorders. In some cases, the data may be used byhealthcare organizations or insurance companies to tailor preventivebehavioral health programs for users, which can help users to improvetheir health and well-being.

The methods and systems described herein can be configured to recordneural activity, and detect corrupt or inaccurate sensoryrepresentations from the neural activity based on information about thereal-world environment. The real-world environmental information can beobtained through the use of one or more sensors mounted on a patient'sbody, backpack, clothing, etc., or implanted.

The methods and systems described herein utilize developments in twodifferent fields: neurotechnology and statistical computation. It isincreasingly possible to measure activity in the nervous system withimpressive accuracy. For instance, with the advent of new high-densitymicroelectrode recording techniques, neural activity can now be sampledusing hundreds of thousands of independent channels. Accordingly,sufficient information relating to ongoing neural activity can becollected, and used to determine if this activity is consistent withexpected sensory representations. Machine learning techniques havedramatically advanced the fields of computer vision and advanced audioprocessing (especially image recognition and speech recognition). Byleveraging the above developments, the methods and systems describedherein can extract high-level representations of audio-visualinformation, that can be used as reference for checking whether thereconstructed brain signals accurately correspond with actual real-worldenvironmental stimuli.

Next, various embodiments of the disclosure will be described withreference to the drawings.

FIG. 1 illustrates a neural data correspondence system, in accordancewith some embodiments. In one aspect, a neural data correspondencesystem 100 may include a neural data analysis module 110, a sensingmodule 120, and a correspondence module 130. Each of the modules 110,120, and 130 may be operatively connected to one another via a networkor any type of communication link that allows the transmission of datafrom one component to another.

As previously described, a patient suffering from mental disorders mayexperience corrupt or inaccurate sensory perceptions. For instance, thepatient may suffer from an auditory or visual hallucination, which is aperception in the absence of external stimuli that has qualities of realperception. Corrupt or inaccurate sensory perceptions can also beassociated with drug use, sleep deprivation, psychosis, and deliriumtremens. Corrupt or inaccurate sensory perceptions can occur in anysensory modality—visual, auditory, olfactory, gustatory, tactile,proprioceptive, equilibrioceptive, nociceptive, thermoceptive,chemoreceptive, and chronoceptive. To a patient, corrupt or inaccuratesensory perceptions can be vivid, substantial, and perceived to belocated in external objective space. For example, auditory and visualhallucinations are common in patients suffering from schizophrenia. Inother cases, the corrupt or inaccurate sensory perceptions may becharacterized by brain activity becoming out of sync with thesurroundings in a meaningful manner. For instance, patients withAlzheimer's disease or other forms of dementia may be characterized by aperception of being at home when a patient is actually in a hospital, orof no longer recognizing a person who should be extremely familiar tothe patient. The correspondence system of FIG. 1 can be used todetermine whether a patient is experiencing corrupt or inaccuratesensory perceptions or the onset of corrupt or inaccurate sensoryperceptions, using real-world sensor data collected via one or moresensors mounted on the patient's body.

The neural data analysis module 110 may be configured to receive neuralsignals. The neural signals may be collected using wires,microelectrodes, optical sensors, magnetic field sensors, or any othersensors that can record the activity the nervous system, or which thebrain is one part. The plurality of wires or microelectrodes may be partof one or more massively parallel neural interface probes that areconfigured to be implanted into one or more areas of a person's brain.The one or more neural interface probes can be configured to recordinformation associated with neural activity and produce an output thatrepresents neural activity. In some embodiments, the neural dataanalysis module may generate a signal representative of a pattern ofneural activity. The neural code may comprise representations of one ormore neural activities associated with a plurality of neural eventsdetected from the neural signals.

In some cases, the neural interface probes may comprise one or moreneural detection channels. For instance, the neural interface probes maycomprise more than 10, more than 100, more than 1000, more than 10000,more than 100000, or more than 1000000 neural detection channels. Theneural detection channels may comprise electrodes, microwires, opticalsensors, magnetic field sensors, or any other sensor. In someembodiments, the one or more neural interface probes may include amicrowire bundle bonded to a CMOS array. In other embodiments, the oneor more neural interface probes may include an array of siliconmicroelectrode probes that are bonded to a CMOS array, such that eachelectrode site is routed to a unique array position. In otherembodiments, the neural interface may consist of scalp electrodes. Inother embodiments, the neural interface may consist of electrodes placedon the surface of the brain or the surface of the dura mater. In otherembodiments, the neural interface may record magnetic or optical signalsfrom the nervous system.

The massively parallel neural interface probes can be implanted intodifferent areas of the brain associated with different sensoryprocessing (e.g., visual, auditory, tactile, taste, smell,position/movement, interoception, etc.). In one example, one or moreneural interface probes may be implanted in an area of the brainassociated with auditory processing, such as the auditory cortex. Inanother example, one or more neural interface probes may be implanted inan area of the brain associated with visual processing, such as theprimary visual cortex or fusiform gyrus.

In yet another example, one or more neural interface probes may beimplanted in an area of the brain associated with spatial or locationalawareness. For instance, the one or more neural interface probes may beimplanted in the hippocampus and configured to record neural signalsfrom “place” cells located in the hippocampus. Alternatively, the one ormore neural interface probes may be implanted in the entorhinal cortexand configured to record neural signals from “grid” cells located in theentorhinal cortex.

The sensing module 120 may include one or more types of sensors that areconfigured to detect signals that mostly emanate from outside thepatient. These signals are indicative of reality, as opposed to thecorrupt or inaccurate sensory perceptions that a patient may beexperiencing. The signals may come from many different modalities,including visual, auditory, tactile, taste, smell, position/movement,and/or interoception.

Examples of types of sensors may include inertial sensors (e.g.accelerometers, gyroscopes, and/or gravity detection sensors, which mayform inertial measurement units (IMUs)), location sensors (e.g. globalpositioning system (GPS) sensors, mobile device transmitters enablinglocation triangulation), heart rate monitors, external temperaturesensors, skin temperature sensors, capacitive touch sensors, sensorsconfigured to detect a galvanic skin response (GSR), vision sensors(e.g. imaging devices capable of detecting visible, infrared, orultraviolet light, such as cameras), proximity or range sensors (e.g.ultrasonic sensors, lidar, time-of-flight or depth cameras), altitudesensors, attitude sensors (e.g. compasses), pressure sensors (e.g.barometers), humidity sensors, vibration sensors, audio sensors (e.g.microphones), and/or field sensors (e.g. magnetometers, electromagneticsensors, radio sensors).

The sensing module may further include one or more devices capable ofemitting a signal into an environment. For instance, the sensing modulemay include an emitter along an electromagnetic spectrum (e.g. visiblelight emitter, ultraviolet emitter, infrared emitter). The sensingmodule may include a laser or any other type of electromagnetic emitter.The sensing module may emit one or more vibrations, such as ultrasonicsignals. The sensing module may emit audible sounds (e.g. from aspeaker). The sensing module may emit wireless signals, such as radiosignals or other types of signals.

In some cases, one or more of the sensors may establish an absolute,relative, or contextual location. For instance, one or more of thesensors may be a global positioning system (GPS) sensor that determinesthe person's longitude and latitude. Such a GPS sensor may provideabsolute location, such as that the person is located at 39.7392° N,104.9903° W. The one or more sensors may be any sensor that determinesan absolute location. As another example, one or more of the sensors maybe a wireless detector that receives a signal from a specific location.Such a detector may provide relative location, such as that the personis located 300 meters north of their house. The one or more sensors maybe any sensor that determines a relative location. As yet anotherexample, one or more of the sensors may be a sensor that providescontextual information, such as that the person is at their place ofwork. The one or more sensors may be any sensor that determines acontextual location.

In some embodiments, the sensing module may be configured to performsensor fusion on different types of sensor data. For example, the sensormodule may include a sensor/receiver for GPS, an inertial measurementunit (IMU), another sensor for cell tower distances, etc. These sensorscan be used to obtain different estimates of a location. The sensorfusion may include the use of an algorithm and a Kalman filter tocombine estimators, that have weakly correlated errors, into a combinedestimator whose error is less than the error of any one of the originalestimators. Accordingly, the different sensors can be used to compensatefor deficiencies/inaccuracies of the other. Alternatively, the data fromthe sensors may be combined and its dimensionality reduced by canonicalcorrelation analysis or an analogous technique (e.g. deep canonicalcorrelation analysis or deep general canonical correlation analysis)focused on finding the correspondence between the multivariate sensordata and the multivariate neural data.

In some embodiments, one or more sensors of the sensing module may beincorporated into a wearable device. Examples of wearable devices mayinclude smartwatches, wristbands, glasses, gloves, headgear (such ashats, helmets, virtual reality headsets, augmented reality headsets,head-mounted devices (HMD), headbands), pendants, armbands, leg bands,shoes, vests, motion sensing devices, etc. The wearable device may beconfigured to be worn on a part of a user's body (e.g. a smartwatch orwristband may be worn on the user's wrist).

The correspondence module can be used to detect corrupt or inaccuratesensory representations. The correspondence module may be configured toreceive (a) neural data 112 from the neural data analysis module 110 and(b) real-world sensor data 122 from the sensing module 120. Thecorrespondence module may further analyze and compare the neural data112 and sensor data 122, so as to generate an output 132 that isindicative of a degree of correspondence between perceptions obtainedfrom the neural data with respect to the real-world environmentalstimuli. The degree of correspondence may indicate whether a patient isexperiencing corrupt or inaccurate sensory perceptions or is accuratelyexperiencing reality. In some cases, the degree of correspondence mayindicate an extent or severity of a patient's corrupt or inaccuratesensory perceptions.

The correspondence module may be implemented anywhere within the neuraldata correspondence system and/or outside of the neural datacorrespondence system. In some embodiments, the correspondence modulemay be implemented on a server. In other embodiments, the correspondencemodule may be implemented on a wearable device described elsewhereherein. In still other embodiments, the correspondence module may beimplemented on a portable electronic device, such as a smartphone,tablet computer, or laptop computer. Additionally, the correspondencemodule may be implemented in the neural data analysis module.Alternatively, the correspondence module may be implemented in thesensor module. In other embodiments, the correspondence module may beimplemented in both the neural data analysis module and the sensormodule, or external to both the neural data analysis module and thesensor module. The correspondence module may be implemented usingsoftware, hardware, or a combination of software and hardware in one ormore of the above-mentioned components within the neural datacorrespondence system.

The correspondence module may be capable of determining a correspondencebetween the neural data 112 and the real-world sensor data 122 usingalgorithms based in part on statistical analysis, information theory,machine learning, signal processing, pattern recognition, and/ordetection theory. The correspondence module may utilize a supervisedmachine learning method, a semi-supervised machine learning method, or asupervised machine learning method known to one having skill in the art.An algorithm may assume a certain statistical model and try to detectwhether a patient is experiencing corrupt or inaccurate sensoryperceptions based on the model. The algorithm may also estimate adifferent statistical model for each patient in a population ofpatients, and use each unique model to determine whether each patient isexperiencing corrupt or inaccurate sensory perceptions.

In some embodiments, the statistical model may be implemented in theform of a neural network. In some embodiments, the statistical model maybe implemented in the form of linear regression, non-linear regression,penalized regression techniques such as LASSO or Ridge regression,orthogonal matching pursuit, mutual information, Bayesian regression,stochastic gradient descent, a passive aggressive algorithm, linear ornon-linear discriminant analysis, kernel regression, support vectormachines, nearest neighbor algorithms, Gaussian processes, crossdecomposition, naïve Bayes, decision tree algorithms and other ensemblemethods including but not limited to Random Forest and gradientboosting, and neural network models including but not limited toperceptrons, convolutional neural networks, and deep neural networks.

The measure of correspondence may be any output of the statisticalmodel, including but not limited to correlation, mutual information, orany discrete or continuous value returned by a function which takes asinput signals from the neural module(s) and or the sensor module(s).

The correspondence may be determined in part via supervised learningalgorithms which occurs either online within the device, or offlineduring the design and manufacture of the device. These supervisedlearning algorithms would use as training data labeled examples oftypical and atypical neural data, labeled examples of sensory data indifferent kinds of environments, and labeled examples of agreements anddisagreements between sensory and neural data. For example, sensory andneural data from actual hallucination episodes, as well as from normalactivity, may be in included in the training set.

The correspondence may be determined using several different features ofneural data, including raw signals, frequency-filtered signals, actionpotentials, local field potentials, or any other signal emitted by thenervous system. They may be processed and analyzed in the time domain orin the frequency domain. They may contain spatial information about thelocations of the neural data probes.

The computation of correspondence may exploit known anatomical orfunctional connectivity patterns in the nervous system. For example, ifit is known that brain area V1 is afferent to brain area V2, then thecorrespondence module may exploit the expectation that data recordedfrom V1 and V2 may have a certain causal relationship, such as aspecific range of time or phase lags. Thusly, deviation from theseexpected values could be an indication of low correspondence.

The correspondence may be determined using several different features ofenvironmental sensor data, including raw signals or frequency-filteredsignals. They may be processed and analyzed in the time domain or in thefrequency domain. They may contain spatial information about thelocations of the sensors and imputed locations of matter or energy inthe environment.

In some embodiments, the correspondence module may include an on-linedecoder and an error detector that are configured to determine a degreeof correspondence between the neural data and the real-world sensordata. In some cases, the correspondence may be based on a statisticalmodel. In some cases, the correspondence may be based on informationtheory, such as the mutual information between the neural data and thereal-world sensor data from one or more real-world sensors. In somecases, the correspondence may be determined by machine learningalgorithms, including but not limited to those that determine whether aset of data are likely to belong to a particular class, for example theclass of hallucination-related neural signals. The decoder may inferaudio signals from the neural data and/or the sensor data. For example,the decoder may include one or more instances of speech recognitionsoftware. The speech recognition software may be used to reconstructspeech and other sounds from neural signals that are recorded by the oneor more neural interface probes implanted in an area of the brainassociated with auditory processing. The one or more sensors maycomprise audio sensors. The speech recognition software may be used toreconstruct speech and other sounds recorded by the audio sensors. Thecorrespondence module may then compare the reconstructed neuralperception of sound to the real audio environment captured by the one ormore audio sensors in order to detect whether the neural representationsof speech and other sounds correspond to real-life audio stimuli, orwhether they correspond to corrupt or inaccurate auditory perceptions.

In some cases, the decoder may infer visual signals from the neural dataand/or the sensor data. For instance, the decoder may include one ormore instances of image recognition software. The image recognitionsoftware may comprise facial recognition, object recognition, or scenesegmentation software. The image recognition software may be used toreconstruct neural representations of visual data that are recorded bythe one or more massively parallel neural interface probes implanted inan area of the brain associated with visual processing. The one or moresensors may comprise image sensors. The image recognition software maybe used to reconstruct images and other visual data recorded by theimage sensors. The reconstructed neural perception of visual data maythen be compared with the real-world image data captured by one or moreimaging devices, in order to detect whether the neural representationsof visual data correspond to real-life visual stimuli, or whether theycorrespond to corrupt or inaccurate visual perceptions.

In some cases, the decoder may infer spatial locations from the neuraldata and/or the sensor data. For instance, the decoder may include oneor more instances of spatial location software. The spatial locationsoftware may be used to reconstruct neural representations of spatiallocation data that are recorded by the one or more massively parallelneural interface probes implanted in an area of the brain associatedwith spatial location processing. The one or more sensor may comprisespatial location sensors. The spatial location software may be used toreconstruct the spatial location of the patient based on signalsrecorded from the spatial location sensors. The reconstructed neuralperception of spatial location may then be compared with the real-worldspatial location data captured by one or more spatial location sensors,in order to detect whether the neural representations of spatiallocation correspond to the real-life location, or whether theycorrespond to faulty cognitive processing.

In some cases, the decoder may compare neural data and sensor data thatare different in nature. For instance, the decoder may reconstructneural representations of speech or other sound that are recorded by theone or more massively parallel neural interface probes. Thereconstructed neural perception of sound data may then be compared withthe real-world image data captured by one or more imaging devices. If,for instance, the reconstructed neural perception indicates that thepatient is hearing a voice, but the image data indicates that no one isin the room with the patient, this may be an indication that the patientis experiencing corrupt or inaccurate sensory representations.Similarly, the decoder may compare any form of neural data with any formof sensor data.

The error detector may be configured to generate an error signal thatrepresents a lack of correspondence. The error signal may reflect only alack of correspondence, or may reflect any other indication that theneural data is unusual and/or reflects a lack of physical or mentalhealth.

The error signal may be indicative of the patient experiencing anepisode of corrupt or inaccurate sensory perceptions. In someembodiments, the decision module may act on the error signal to actuatea visual signal and/or an audio signal to the patient to indicate thatan error is detected. The visual signal may comprise a flashing light,and the audible signal may comprise an audible alarm. The visual and/oraudio signals may be used to notify the patient that he/she ishallucinating, and to prompt the patient to revert back to reality. Thesignals may be produced externally to the patient or may signal thepatient through neural stimulation. For instance, the signals may evokethe perception of a beep rather than producing a beep that would beaudible to others.

The output from the decoder may be transmitted to allow remotemonitoring. For instance, the output from the decoder may be transmittedto a patient's doctor or hospital for remote monitoring. The remotemonitoring may allow the patient's doctor or hospital to note when thepatient has experienced sensory data that does not align with reality.The remote monitoring may allow the patient's doctor or hospital tointervene. For instance, the remote monitoring may allow the patient'sdoctor or hospital to call the patient for a follow-up visit, or to senda reminder to the patient to take medication intended to lessen thehallucinatory effects of the patient's condition. The transmission ofthe output from the decoder may be via any wired communication means orwireless communication means. The wired communication may be viaEthernet. The wireless communication may be via Wi-Fi. The wirelesscommunication may be via a cellular network. The wireless communicationmay be via a cellular networking protocol such as Global System forMobile Communications (GSM), General Packet Radio Service (GPRS), CodeDivision Multiple Access (CDMA), CDMA 2000, Evolution Data Maximized(EV-DO), Frequency Division Multiple Access (FDMA), Time DivisionMultiple Access, 1st generation (1G), 2nd generation (2G), 3rdgeneration (3G), 4th generation (4G), 5th generation (5G), or any othercellular networking protocol. The wireless communication may be via anInstitute of Electrical and Electronics Engineers (IEEE) wirelessprotocol, such as 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac,802.11ad, 802.11af, 802.11ah, 802.11ai, 802.11aj, 802.11aq, 802.11ax,802.11ay, or any other IEEE wireless protocol. The wirelesscommunication may be a via a Bluetooth wireless protocol. The wirelesscommunication may be via any IEEE standard for wired or wirelesscommunication.

The ability to detect when the brain is misprocessing or even creatingsensory percepts would allow for better treatment of the conditionsunderlying these symptoms. For instance, detecting a hallucinationassociated with chronic schizophrenia could trigger an automatic dose ofmedication or would allow for remote monitoring of patients that do notelect for institutional care. As an example, a healthcare provider canuse information, including the degree of correspondence, to assist thepatient in managing the mental disorders through counseling, medication,electrical neural stimuli, etc.

FIG. 2 illustrates a closed-loop system comprising a neural stimulator,in accordance with some embodiments. The system 200 may comprise theneural data analysis module 110 that outputs neural data 112, thesensing module 120 that outputs real-world sensor data 122, thecorrespondence module that outputs a degree of correspondence 132, andthe decision module. The system may further comprise a neural stimulator140.

The output 132 may be provided to the neural stimulator. In some cases,the neural stimulator may be a medical device (such as an implantedmedical device) that is capable of automatically dispensing apredetermined dosage of medication based on the output 132 (i.e. whetherthe patient is or is not experiencing corrupt or inaccurate sensoryperceptions). Alternatively, the neural stimulator may be configured tovary a dispensed dosage to the patient depending on the degree ofcorrespondence (i.e. the extent and/or degree of corrupt or inaccuratesensory perceptions).

The output 132 may be a signal to communicate with another biologicaldevice. For example, it may be a signal to change the amplitude orfrequency of a deep-brain stimulation device separately implanted in thepatient.

In some embodiments, the neural stimulator may be implemented on one ormore massively parallel neural interface probes. Such one or moremassively parallel neural interface probes may include an electrodearray that is capable of interrupting hallucinatory (pathological)precepts, in addition to being able to stimulate and record neuralactivity. The neural stimulator may be configured to impart a change inthe mental status of the patient. Such a change may be detected by theneural data analysis module, which may give rise to altered neural dataand an altered degree of correspondence. This may have the effect ofaltering the action taken by the neural stimulator. In this manner, thesystem 200 may comprise a feedback loop between the neural data and theaction taken by the neural stimulator.

The output from the decoder may be used to trigger the release oradministration of a pharmaceutical. The pharmaceutical may be apharmaceutical prescribed by the patient's doctor to lessen thehallucinatory effects of the patient's condition. The decoder may becommunicatively coupled with a device that administers thepharmaceutical. The coupling may be via any wired communication means orwireless communication means. The wireless communication may be viaBluetooth. The wireless communication may be via a cellular network. Thedevice may administer the pharmaceutical intravenously. The device mayadminister the pharmaceutical intramuscularly. The device may administerthe pharmaceutical transdermally. The device may administer thepharmaceutical orally. The device may administer the pharmaceuticalintracranially, intraventricularly, or intraperitoneally.

The pharmaceutical may be a pharmaceutical intended to treat effects ofpsychosis disorders, such as schizophrenia. For instance, thepharmaceutical may be any of the following: chlorprothixene,levomepromazine, perazine, promethazine, prothipendyl, sulpiride,thioridazine, zuclopenthixol, perphenazine, benperidol, bromperidol,fluphenazine or its salts, fluspirilen, haloperidol, pimozide,amisulpride, aripiprazole, asenapine, chlorpromazine, clozapine,flupenthixol, iloperidone, melperone, olanzapine, paliperidone,penfluridol, quetiapine, risperidone, sertindole, thiothixene,trifluoperazine, ziprasidone, zotepine, or pericyazine. Thepharmaceutical may be any other pharmaceutical intended to treat theeffects of psychosis disorders.

The pharmaceutical may be intended to treat the effects of idiopathicpain. For instance, the pharmaceutical may be any of the following: anon-steroidal anti-inflammatory drug (such as paracetamol,acetaminophen, aspirin, naproxen, or ibuprofen), an opioid (such ascodeine, morphine, or fentanyl), a muscle relaxant (such as diazepam),an anti-convulsant (such as gabapentin or pregabalin), ananti-depressant (such as amitriptyline), or a steroid. Thepharmaceutical may be any pharmaceutical that alleviates idiopathicpain.

The pharmaceutical may be a pharmaceutical intended to treat effects ofAlzheimer's disease or other forms of dementia. For instance, thepharmaceutical may be an acethylcholinesterase inhibitor, such astacrine, rivastigmine, galantamine, or donepezil. The pharmaceutical maybe a N-methyl-D-asparate (NMDA) receptor antagonist, such as memantine.The pharmaceutical may be any pharmaceutical intended to treat effectsof Alzheimer's disease or other forms of dementia. The pharmaceuticalmay be any pharmaceutical intended to treat the effects on any otherpsychiatric disorder.

FIG. 3 illustrates the communication between different body-mountedsensors, an implanted neural interface probe, a neural data analysismodule, and a correspondence module, in accordance with someembodiments. The neural analysis module 110 may be implanted into thepatient's brain in any of the manners described above. The neural dataanalysis may be communicatively coupled to sensing modules 120-1 and/or120-2. The sensing module 120-1 may be a component of a mobile devicecarried by the patient, which may include one or more sensors such ascameras, microphones, accelerometers, gyroscopes, compasses, GPS, etc.The sensing module 120-2 may be a component of a wrist-wearable devicesuch as a smartwatch or wristband, which may include on or more sensorsfor measuring body temperature, heart rate, location, motion of thewrist, etc. The neural analysis module 110 and sensing modules 120-1and/or 120-2 may be communicatively couple to the correspondence module130. The correspondence module may be an app or other program held inmemory on the mobile device and/or wrist-wearable device. Thecorrespondence module may be peripheral hardware componentscommunicatively coupled to the mobile device and/or wrist-wearabledevice. The neural stimulator 140 may also be implanted into thepatient's brain in the manners described above.

FIG. 4 illustrates the reconstruction of perceptions from neural dataand correspondence of the perceptions with real-world sensor data, inaccordance with some embodiments. The neural data 112 may be the outputfrom one or more massively parallel neural interface probes, asdescribed above. The neural data may comprise one or more of visualrepresentations (e.g. those representations arising from areas of thebrain associated with vision processing, such as the primary visualcortex or fusiform gyms) 112-1, audio representations (e.g. thoserepresentations arising from areas of the brain associated with auditoryprocessing, such as the auditory cortex) 112-2, or tactilerepresentations (e.g. those representations arising from areas of thebrain associated with spatial or locational awareness, such as the“place” cells of the hippocampus or the “grid” cells of the entorhinalcortex) 112-3.

The neural data may be passed to the decoder 116. The decoder mayprocess the neural data to form one or more reconstructed signals. Forinstance, the visual representations 112-1 may be processed to formreconstructed images 116-1. The audio representations 112-2 may beprocessed to form reconstructed audio signals 116-2. The tactilerepresentations may be processed to form reconstructed tactile signals116-3. Any of these representations may be processed alone or incombination with other representations.

The real-world sensor data 122 may be the output from one or moresensors, as described above. The sensor data may comprise one or more ofvisual sensor data (e.g. data from imaging devices capable of detectingvisible, infrared, or ultraviolet, such as cameras) 122-1, audio sensordata (e.g. data from audio sensors such as microphones) 122-2, or touchsensor data (e.g. data from touch sensors such as capacitive touchsensors) 122-3. One or more of the reconstructed images, reconstructedaudio signals, or reconstructed tactile signals may be sent to thecorrespondence module 130. Similarly, one or more of the visual sensordata, audio sensor data, or touch sensor data may be sent to thecorrespondence module. The correspondence module may act to determine acorrespondence between the decoded neural data 116 and the real-worldsensor data 122 using algorithms based in part on statistical analysis,information theory, machine learning, signal processing, patternrecognition, and/or detection theory. An algorithm may assume a certainstatistical model and try to detect whether a patient is experiencingcorrupt or inaccurate sensory perceptions based on the model.Alternatively, the algorithm may assume no model and simply compare thesensory and neural data using machine learning approaches.

FIG. 5 shows an exemplary method 500 by which neural activity may berecorded and compared to data obtained from a sensor, allowing for thedetection of corrupt or inaccurate sensory representations. The method500 comprises obtaining neural data; obtaining sensor data; determininga correspondence between the sensor data and the neural data;determining whether a perception error is less than a predeterminedthreshold; and, if the perception error is too high: concluding that theuser may be experiencing inaccurate neural perceptions; generating aperception error signal; and, optionally, applying neural stimuli tocorrect the perception error. In some embodiments, method 500 may beperformed automatically by a processor associated with a computerreadable memory.

In step 502, neural data is obtained by a neural data analysis module.The neural signals may be collected using a plurality of wires ormicroelectrodes that have been implanted into deep neural matter orsuperficial neural matter. The plurality of wires or microelectrodes maybe part of one or more massively parallel neural interface probes thatare configured to be implanted into one or more areas of a person'sbrain. The one or more neural interface probes can be configured torecord information associated with neural activity and produce an outputthat represents neural activity. In some embodiments, the neural dataanalysis module may generate an analog or digital signal correspondingto neural activity patterns. The neural code may compriserepresentations of one or more neural activities associated with aplurality of neural events detected from the neural signals.

In some cases, the neural interface probes may comprise one or moreneural detection channels. For instance, the neural interface probes maycomprise more than 10, more than 100, more than 1000, more than 10000,more than 100000, or more than 1000000 neural detection channels. Theneural detection channels may comprise electrodes, microwires, opticalsensors, magnetic field sensors, or any other sensor. In someembodiments, the one or more neural interface probes may include amicrowire bundle bonded to a CMOS array. In other embodiments, the oneor more neural interface probes may include an array of siliconmicroelectrode probes that are bonded to a CMOS array, such that eachelectrode site is routed to a unique array position. In otherembodiments, the neural interface may consist of scalp electrodes. Inother embodiments, the neural interface may consist of electrodes placedon the surface of the brain or the surface of the dura mater. In otherembodiments, the neural interface may record magnetic or optical signalsfrom the nervous system.

The massively parallel neural interface probes can be implanted intodifferent areas of the brain associated with different sensoryprocessing (e.g., visual, auditory, tactile, taste, smell,position/movement, interoception, etc.). In one example, one or moreneural interface probes may be implanted in an area of the brainassociated with auditory processing, such as the auditory cortex. Inanother example, one or more neural interface probes may be implanted inan area of the brain associated with visual processing, such as theprimary visual cortex or fusiform gyrus.

In yet another example, one or more neural interface probes may beimplanted in an area of the brain associated with spatial or locationalawareness. For instance, the one or more neural interface probes may beimplanted in the hippocampus and configured to record neural signalsfrom “place” cells located in the hippocampus. Alternatively, the one ormore neural interface probes may be implanted in the entorhinal cortexand configured to record neural signals from “grid” cells located in theentorhinal cortex.

In step 504, sensor data is obtained by a sensor module. The sensor datamay be obtained from one or more types of sensors that are configured todetect real-world environmental stimuli. The real-world environmentalstimuli are indicative of reality, as opposed to the corrupt orinaccurate sensory perceptions perceptions that a patient may beexperiencing. The real-world environmental stimuli may be provided inthe form of visual, auditory, tactile, taste, smell, position/movement,and/or interoception.

Examples of types of sensors may include inertial sensors (e.g.accelerometers, gyroscopes, and/or gravity detection sensors, which mayform inertial measurement units (IMUs)), location sensors (e.g. globalpositioning system (GPS) sensors, mobile device transmitters enablinglocation triangulation), heart rate monitors, external temperaturesensors, skin temperature sensors, capacitive touch sensors, sensorsconfigured to detect a galvanic skin response (GSR), vision sensors(e.g. imaging devices capable of detecting visible, infrared, orultraviolet light, such as cameras), proximity or range sensors (e.g.ultrasonic sensors, lidar, time-of-flight or depth cameras), altitudesensors, attitude sensors (e.g. compasses), pressure sensors (e.g.barometers), humidity sensors, vibration sensors, audio sensors (e.g.microphones), and/or field sensors (e.g. magnetometers, electromagneticsensors, radio sensors).

The sensing module may further include one or more devices capable ofemitting a signal into an environment. For instance, the sensing modulemay include an emitter along an electromagnetic spectrum (e.g. visiblelight emitter, ultraviolet emitter, infrared emitter). The sensingmodule may include a laser or any other type of electromagnetic emitter.The sensing module may emit one or more vibrations, such as ultrasonicsignals. The sensing module may emit audible sounds (e.g. from aspeaker). The sensing module may emit wireless signals, such as radiosignals or other types of signals.

In some cases, one or more of the sensors may establish an absolute,relative, or contextual location. For instance, one or more of thesensors may be a global positioning system (GPS) sensor that determinesthe person's longitude and latitude. Such a GPS sensor may provideabsolute location, such as that the person is located at 39.7392° N,104.9903° W. The one or more sensors may be any sensor that determinesan absolute location. As another example, one or more of the sensors maybe a wireless detector that receives a signal from a specific location.Such a detector may provide relative location, such as that the personis located 300 meters north of their house. The one or more sensors maybe any sensor that determines a relative location. As yet anotherexample, one or more of the sensors may be a sensor that providescontextual information, such as that the person is at their place ofwork. The one or more sensors may be any sensor that determines acontextual location.

In some embodiments, one or more sensors of the sensing module may beincorporated into a wearable device. Examples of wearable devices mayinclude smartwatches, wristbands, glasses, gloves, headgear (such ashats, helmets, virtual reality headsets, augmented reality headsets,head-mounted devices (HMD), headbands), pendants, armbands, leg bands,shoes, vests, motion sensing devices, etc. The wearable device may beconfigured to be worn on a part of a user's body (e.g. a smartwatch orwristband may be worn on the user's wrist).

Steps 502 and 504 may be performed substantially in real-time.Additionally, steps 502 and 504 may be performed in parallel.

In step 506, a correspondence is determined between the sensor data andthe neural data by a correspondence module. The correspondence modulecan be used to detect corrupt or inaccurate sensory representations. Thecorrespondence module may be configured to receive (a) neural data fromthe neural data analysis module and (b) real-world sensor data from thesensing module. The correspondence module may further analyze andcompare the neural data and sensor data, so as to generate an outputthat is indicative of a degree of correspondence between perceptionsobtained from the neural data with respect to the real-worldenvironmental stimuli. The degree of correspondence may indicate whethera patient is experiencing corrupt or inaccurate sensory perceptions oris accurately experiencing reality. In some cases, the degree ofcorrespondence may indicate an extent or severity of a patient's corruptor inaccurate sensory perceptions.

The correspondence module may be implemented anywhere within the neuraldata correspondence system and/or outside of the neural datacorrespondence system. In some embodiments, the correspondence modulemay be implemented on a server. In other embodiments, the correspondencemodule may be implemented on a wearable device described elsewhereherein. In still other embodiments, the correspondence module may beimplemented on a portable electronic device, such as a smartphone,tablet computer, or laptop computer. Additionally, the correspondencemodule may be implemented in the neural data analysis module.Alternatively, the correspondence module may be implemented in thesensor module. In other embodiments, the correspondence module may beimplemented in both the neural data analysis module and the sensormodule, or external to both the neural data analysis module and thesensor module. The correspondence module may be implemented usingsoftware, hardware, or a combination of software and hardware in one ormore of the above-mentioned components within the neural datacorrespondence system.

The correspondence module may be capable of determining a correspondencebetween the neural data and the real-world sensor data using algorithmsbased in part on statistical analysis, information theory, machinelearning, signal processing, pattern recognition, and/or detectiontheory. An algorithm may assume a certain statistical model and try todetect whether a patient is experiencing corrupt or inaccurate sensoryperceptions based on the model. The statistical model may be based onBayesian statistics. The algorithm may also estimate a differentstatistical model for each patient in a population of patients, and useeach unique model to determine whether each patient is experiencingcorrupt or inaccurate sensory perceptions. In some embodiments, thestatistical model may be implemented in the form of a Bayesian model. Insome embodiments, the implementation may be via machine learningalgorithms.

In some embodiments, the correspondence module may include algorithms todetermine a degree of correspondence between the neural data and thereal-world sensor data based on a statistical model, information theory,or other machine learning approach. The algorithms may take as inputsfeatures selected from the neural data and/or the sensor data, such asaudio-related signals. For example, the decoder may include one or moreinstances of speech recognition software. The speech recognitionsoftware may be used to reconstruct neural representations of speech andother sounds that are recorded by the one or more massively parallelneural interface probes implanted in an area of the brain associatedwith auditory processing. The one or more sensors may comprise audiosensors. The speech recognition software may be used to reconstructspeech and other sounds recorded by the audio sensors. Thecorrespondence module may then compare the reconstructed neuralperception of sound to the real audio environment captured by the one ormore audio sensors in order to detect whether the neural representationsof speech and other sounds correspond to real-life audio stimuli, orwhether they correspond to corrupt or inaccurate auditory perceptions.

In some cases, the correspondence module may accomplish the same goal asin the previous paragraph, but for visual signals, and via imagerecognition software. The image recognition software may comprise facialrecognition, object recognition, or scene segmentation software. Theimage recognition software may be used to reconstruct neuralrepresentations of visual data that are recorded by the one or moremassively parallel neural interface probes implanted in an area of thebrain associated with visual processing. The one or more sensors maycomprise image sensors. The image recognition software may be used toreconstruct images and other visual data recorded by the image sensors.The reconstructed neural perception of visual data may then be comparedwith the real world image data captured by one or more imaging devices,in order to detect whether the neural representations of visual datacorrespond to real-life visual stimuli, or whether they correspond tocorrupt or inaccurate visual perceptions.

In some cases, the decoder may accomplish the same goal as in theprevious paragraph, but for spatial locations and via spatial locationsoftware. The spatial location software may be used to reconstructneural representations of spatial location data that are recorded by theone or more massively parallel neural interface probes implanted in anarea of the brain associated with spatial location processing. The oneor more sensor may comprise spatial location sensors. The spatiallocation software may be used to reconstruct the spatial location of thepatient based on signals recorded from the spatial location sensors. Thereconstructed neural perception of spatial location may then be comparedwith the real-world spatial location data captured by one or morespatial location sensors, in order to detect whether the neuralrepresentations of spatial location correspond to the real-lifelocation, or whether they correspond to faulty cognitive processing.

In some cases, the correspondence module may compare neural data andsensor data from different modalities. For instance, the decoder mayreconstruct neural representations of speech or other sound that arerecorded by the one or more massively parallel neural interface probes.The reconstructed neural perception of sound data may then be comparedwith the real-world image data captured by one or more imaging devices.If, for instance, the reconstructed neural perception indicates that thepatient is hearing a voice, but the image data indicates that no one isin the room with the patient, this may be an indication that the patientis experiencing corrupt or inaccurate sensory representations.Similarly, the decoder may compare any form of neural data with any formof sensor data.

In step 508, it is determined whether the error signal arising in partfrom the correspondence module is sufficient to execute an action. Theerror signal may be related to the degree of agreement between theneural data and the sensor data. Alternatively it may be related to someproperty of the neural data alone. If the error signal is low, this maybe taken to be indicative that the user's neural perceptions match thoseof the real-world environment. If the perception error is large, thismay be taken to be indicative that the user may be experiencinginaccurate neural perceptions. Error signals of sufficientmagnitude—which may depend on context such as patient identity, patienthistory, patient biomarkers, clinician preferences, or othercovariates—will trigger an action signal from the decision module. Insome embodiments, a receiver operating characteristic (ROC) curve can becreated based on a magnitude of the error signal to provide asensitivity/specificity report. The ROC curve is a tool for diagnostictest evaluation, in which the true positive rate (sensitivity) isplotted in function of the false positive rate (specificity) fordifferent cut-off points of a parameter. Each point on the ROC curverepresents a sensitivity/specificity pair corresponding to a particulardecision threshold. The area under the ROC curve (AUC) is a measure ofhow well a parameter can distinguish between two diagnostic groups(hallucination versus normal perceptions).

In step 510, the decision module generates an action signal if the errorsignal is too high in step 508. The error signal may be too high whenthe neural data and the real-world sensor data fail to meet apredetermined correspondence threshold. Alternatively, the error signalmay be high due to properties of the neural data alone. The error signalmay be indicative of the patient experiencing an episode of corrupt orinaccurate sensory perceptions. In some embodiments, the action signalmay be an output that triggers another device to execute a correspondingaction. For instance, the action signal may trigger a visual signaland/or an audio signal to the patient to indicate that an error isdetected. The visual signal may comprise a flashing light, and theaudible signal may comprise an audible alarm. The visual and/or audiosignals may be used to notify the patient that he/she is experiencingcorrupt or inaccurate sensory perceptions, and to prompt the patient torevert back to reality. The signals may be produced externally to thepatient or may signal the patient through neural stimulation. Forinstance, the signals may evoke the perception of a beep rather thanproducing a beep that would be audible to others.

The action signal may be transmitted to allow remote monitoring. Forinstance, the output from the decision module may be transmitted to apatient's doctor or hospital for remote monitoring. The remotemonitoring may allow the patient's doctor or hospital to note when thepatient has experienced sensory data that does not align with reality.The remote monitoring may allow the patient's doctor or hospital tointervene. For instance, the remote monitoring may allow the patient'sdoctor or hospital to call the patient for a follow-up visit, or to senda reminder to the patient to take medication intended to lessen thehallucinatory effects of the patient's condition. The transmission ofthe output from the decoder may be via any wired communication means orwireless communication means. The wired communication may be viaEthernet. The wireless communication may be via Wi-Fi. The wirelesscommunication may be via a cellular network.

In step 512, neural stimuli may be applied to correct the perceptionerror. The ability to detect when the brain is misprocessing or evencreating sensory percepts would allow for better treatment of theconditions underlying these symptoms. For instance, detecting ahallucination associated with chronic schizophrenia could trigger anautomatic dose of medication or would allow for remote monitoring ofpatients that do not elect for institutional care. As an example, ahealthcare provider can use information, including any intermediate orfinal measure in the system, including any computed measure ofcorrespondence, any aspect of the error signal, or the decision signal,to assist the patient in managing the mental disorders throughcounseling, medication, electrical neural stimuli, etc.

The neural stimuli may be applied by neural data correspondence systemcomprising a neural stimulator, in accordance with some embodiments. Thesystem may comprise the neural data analysis module that outputs neuraldata, the sensing module that outputs real-world sensor data, and thecorrespondence module that outputs the correspondence. The system mayfurther comprise a neural stimulator. The output may be provided to theneural stimulator. In some cases, the neural stimulator may be animplanted medical device that is capable of automatically dispensing apredetermined dosage of medication based on the output (i.e. whether thepatient is or is not experiencing corrupt or inaccurate sensoryperceptions). Alternatively, the neural stimulator may be configured tovary a dispensed dosage to the patient depending on the correspondence(i.e. the extent and/or degree of corrupt or inaccurate sensoryperceptions).

In some embodiments, the neural stimulator may be implemented on one ormore massively parallel neural interface probes. Such one or moremassively parallel neural interface probes may include an electrodearray that is capable of interrupting hallucinatory (pathological)precepts, in addition to being able to stimulate and record neuralactivity. The neural stimulator may be configured to impart a change inthe mental status of the patient. Such a change may be detected by theneural data analysis module, which may give rise to altered neural dataand an altered degree of correspondence. This may have the effect ofaltering the action taken by the neural stimulator. In this manner, thesystem may comprise a feedback loop between the neural data, the sensorydata, the correspondence module, and the decision module, and the actiontaken by the neural stimulator.

The output from the decoder may be used to trigger the release oradministration of a pharmaceutical. The pharmaceutical may be apharmaceutical prescribed by the patient's doctor to lessen thehallucinatory effects of the patient's condition. The decoder may becommunicatively coupled with a device that administers thepharmaceutical. The coupling may be via any wired communication means orwireless communication means. The wireless communication may be viaBluetooth. The wireless communication may be via a cellular network. Thedevice may administer the pharmaceutical intravenously. The device mayadminister the pharmaceutical intramuscularly. The device may administerthe pharmaceutical transdermally. The device may administer thepharmaceutical orally.

The pharmaceutical may be a pharmaceutical intended to treat effects ofpsychosis disorders, such as schizophrenia. For instance, thepharmaceutical may be any of the following: chlorprothixene,levomepromazine, perazine, promethazine, prothipendyl, sulpiride,thioridazine, zuclopenthixol, perphenazine, benperidol, bromperidol,fluphenazine or its salts, fluspirilen, haloperidol, pimozide,amisulpride, aripiprazole, asenapine, chlorpromazine, clozapine,flupenthixol, iloperidone, melperone, olanzapine, paliperidone,penfluridol, quetiapine, risperidone, sertindole, thiothixene,trifluoperazine, ziprasidone, zotepine, or pericyazine. Thepharmaceutical may be any other pharmaceutical intended to treat theeffects of psychosis disorders.

The pharmaceutical may be intended to treat the effects of idiopathicpain. For instance, the pharmaceutical may be any of the following: anon-steroidal anti-inflammatory drug (such as paracetamol,acetaminophen, aspirin, naproxen, or ibuprofen), an opioid (such ascodeine, morphine, or fentanyl), a muscle relaxant (such as diazepam),an anti-convulsant (such as gabapentin or pregabalin), ananti-depressant (such as amitriptyline), or a steroid. Thepharmaceutical may be any pharmaceutical that alleviates idiopathicpain.

The pharmaceutical may be a pharmaceutical intended to treat effects ofAlzheimer's disease or other forms of dementia. For instance, thepharmaceutical may be an acethylcholinesterase inhibitor, such astacrine, rivastigmine, galantamine, or donepezil. The pharmaceutical maybe a N-methyl-D-asparate (NMDA) receptor antagonist, such as memantine.The pharmaceutical may be any pharmaceutical intended to treat effectsof Alzheimer's disease or other forms of dementia. The pharmaceuticalmay be any pharmaceutical intended to treat the effects on any otherpsychiatric disorder.

A person of ordinary skill in the art will recognize many variations,alterations and adaptations based on the disclosure provided herein. Forexample, the order of the steps of the method can be changed, some ofthe steps removed, some of the steps duplicated, and additional stepsadded as appropriate. Some of the steps may comprise sub-steps. Some ofthe steps may be automated and some of the steps may be manual. Theprocessor as described herein may comprise one or more instructions toperform at least a portion of one or more steps of the method 500.

The method 500 may operate in a manner that does not require completeunderstanding of the mechanisms by which neural processes representsensory data. Current research is just beginning to probe how neuralsignals give rise to simple behaviors. It will take many years ofresearch activity to arrive at an understanding of neural processes thatallows perfect interpretation of brain activity, if such anunderstanding is even possible. It may also not be necessary that themethod provide complete, accurate decoding of neural representations.

FIG. 6 illustrates a system for monitoring neural activity of a livingsubject, in accordance with some further embodiments. The living subjectmay be a human being (e.g. a patient). In one aspect, system 600 maycomprise one or more neural modules 102 (e.g. 102-1 through 102-n), oneor more sensing modules 120 (e.g. 120-1 through 120-n), one or more datastorage modules 150 (e.g. 150-1 through 150-n), a correspondence module130, a decision module 160, and one or more action modules 170 (e.g.170-1 through 170-n). Each of the above may be operatively connected toone another via a network or any type of communication link that allowsthe transmission of data from one module or component to another.

The neural modules 102 may comprise any of the neural data analysismodules or devices described elsewhere herein. The neural modules maycomprise the same type or different types of devices. The neural modulesmay include one or more devices configured to collect a plurality ofsignals from the nervous system of a living subject. The neural modulesmay include electroencephalography (EEG), electrocorticography (ECoG),intracranial electroencephalography (iEEG), microelectrode arrays, nervecuffs, functional near infrared imaging, functional magnetic resonanceimaging (fMRI), calcium imaging using genetically encoded calciumindicators or voltage sensitive proteins, and any other types ofapparatus or technology that can be used to collect data from thenervous system of the living subject. One or more of the neural modulesmay be used in conjunction with the neural data analysis moduledescribed elsewhere herein.

The sensing modules 120 may include any of the sensing modules orsensors described elsewhere herein. In some embodiments, the sensingmodules or sensors may include image sensors, microphones, inertialsensors such as accelerometers or gyroscopes, GPS, force sensors,capacitive touch sensors, temperature or thermal sensors, chemicalsensors, and so forth.

The data storage modules 150 may include one or more memory devicesconfigured to store data. Additionally or optionally, the data storagemodules 150 may, in some embodiments, be implemented as a computersystem with a storage device. The data storage modules 150 can beconfigured to store statistical prior data for one or more of the neuralmodules 102. The data storage modules 150 can also be configured tostore statistical prior data for one or more of the neural modules 102and their correspondences to the sensor data collected by the sensingmodules 120.

The correspondence module 130 may include one or more embodiments of thecorrespondence module as described elsewhere herein. The decision module160 may include one or more embodiments of the decision module asdescribed elsewhere herein.

The action modules 170 may be configured to receive input from thedecision module 160. Examples of action modules may include computingdevices 170-1, therapeutic delivery devices 170-2, notification devices170-3, and so forth. The computing devices and/or notification devicescan be configured to provide alerts to the living subject (e.g.,patient), and/or to other parties such as healthcare providers oremergency personnel. The notification devices can generate a visualsignal and/or an audio signal to the patient to indicate presence,absence, or extent of a potential cognitive or physiological disturbanceof the patient. The visual signal may comprise a flashing light, and theaudible signal may comprise an audible alarm. For example, the visualand/or audio signals may be used to notify the patient that he/she ishallucinating, and to prompt the patient to revert back to reality. Thesignals may be produced externally to the patient or may signal thepatient through neural stimulation. For instance, the signals may evokethe perception of a beep rather than producing a beep that would beaudible to others. The therapeutic delivery devices can be used todeliver electrical simulation to the patient and/or providepharmaceutical intervention (e.g. applying a medication dosage asdescribed elsewhere herein).

In some embodiments, the correspondence module 130 may be configured tobe in communication with (1) a neural module 102 and (2) one or moreadditional modules comprising a sensing module 120, another neuralmodule 102, and/or a data storage module 150. The neural module(s) maybe configured to collect neural data indicative of perceptionsexperienced by the living subject. The sensing module may be configuredto collect (1) sensor data indicative of real-world information about anenvironment around the living subject, and/or (2) sensor data indicativeof a physical state or physiological state of the living subject. Thedata storage module can be configured to store prior neural data and/orprior sensor data. The correspondence module can be configured tomeasure a correspondence (a) between the neural data collected by theneural module(s) and the sensor data collected by the sensing module,(b) between the neural data collected by two or more neural modules,and/or (c) between the neural data collected by the neural module(s) andthe prior data stored in data storage module. The measuredcorrespondence can be used, for example by the decision module 160, todetermine a presence, absence, or extent of a potential cognitive orphysiological disturbance of the living subject.

In some embodiments, the measured correspondence may comprise a level ofagreement or disagreement (a) between the neural data collected by theneural module(s) and the sensor data collected by the sensing module,(b) between the neural data collected by two or more neural modules,and/or (c) between the neural data collected by the neural module(s) andthe prior data stored in data storage module. The level of agreement ordisagreement need not be binary, and can be measured along a continuum.In some cases, the level of agreement or disagreement may comprisemultiple discrete levels. In some embodiments, the level of agreement ordisagreement may be provided in the form of a multivariate vector. Thecorrespondence module can be measure the correspondence usingstatistical models, information theory, or machine learning algorithms.

In some embodiments, the correspondence module can be configured tomeasure the correspondence (b) between the neural data collected by thetwo or more neural modules, without requiring or utilizing some or allof the sensor data collected by the sensing module. Additionally oroptionally, the correspondence module can be configured measure thecorrespondence (c) between the neural data collected by the neuralmodule(s) and the prior data stored in data storage module, withoutrequiring or utilizing some or all of the sensor data collected by thesensing module.

The decision module 160 can be configured to determine the presence,absence, or extent of the potential cognitive or physiologicaldisturbance based on the measured correspondence output by thecorrespondence module 130. The decision module can be further configuredto generate one or more alerts, notifications, control signals, etc. tothe action modules 170. For example, alerts or notifications can be sentto the patient or other parties such as healthcare providers, suchalerts or notifications being indicative of the presence, absence, orextent of the potential cognitive or physiological disturbance. In somecases, the control signals can be sent to the therapeutic deliverydevices, for example to deliver electrical simulation to the patientand/or provide pharmaceutical intervention (e.g. applying a medicationdosage) to mitigate or counter the potential cognitive or physiologicaldisturbance.

The systems and methods described herein may operate in a manner thatdoes not require complete understanding of the mechanisms by whichneural processes represent sensory data. Current research is justbeginning to probe how neural signals give rise to simple behaviors. Itwill take many years of research activity to arrive at an understandingof neural processes that allows perfect interpretation of brainactivity, if such an understanding is even possible. It may also not benecessary that the method provide complete, accurate decoding of neuralrepresentations. For instance, the systems and methods described hereinmay not require an understanding of neural activity at all, or of thesensory environment at all. The systems and methods may only requirethat neural signals and sensor signals be measureable and representablein an analog or digital form.

All that is required for the systems and methods to detecthallucinations or other corrupt or inaccurate sensory representations isthat the output from the one or more massively parallel neural interfaceprobes be correlated with the data from the one or more sensors. Thiscreates a model (a prior hypothesis) against which all subsequent datafrom both the neural recording system and external sensors can bechecked. This model may be updated based on advances in theunderstanding of neural processes or advances in methods and systems forprocessing neural signals. Regardless of the model, whenever the neuraldata and sensor data suddenly diverge, it may be concluded that there isa high probability (according to the model) that a hallucination orother pathological state has occurred.

For instance, the systems and method need not be able to accuratelyreconstruct the details of human faces from neural representations ofvisual data. It may be sufficient to simply determine that a patient isseeing some face from the neural representation of visual data. If theneural representation and the sensor data disagree about the presence ofa face in the room, this may be taken as an indication that the patientis suffering a visual hallucination.

While preferred embodiments of the present invention have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. Numerousvariations, changes, and substitutions will now occur to those skilledin the art without departing from the invention. It should be understoodthat various alternatives to the embodiments of the invention describedherein may be employed in practicing the invention. It is intended thatthe following claims define the scope of the invention and that methodsand structures within the scope of these claims and their equivalents becovered thereby.

1. A system for monitoring neural activity, comprising: a neural dataanalysis module configured to extract neural data from a plurality ofsignals that are collected from the nervous system of a living subjectusing one or more devices; a sensing module configured to collect sensordata indicative of real-world signals from an environment around orwithin the living subject; and a correspondence module in communicationwith the neural data analysis module and the sensing module, wherein thecorrespondence module is configured to generate an output based on theneural data and the sensor data, wherein the output is indicative of acorrespondence between the environment and neural perceptionsexperienced by the living subject.
 2. The system of claim 1, wherein thecorrespondence module is configured to detect corrupt or inaccuratesensory representations from the correspondence based on statisticalmodels, information theory, or machine learning techniques.
 3. Thesystem of claim 1, where the plurality of signals collected from thenervous system comprises electrical signals, magnetic signals, and/oroptical signals.
 4. The system of claim 1, wherein the one or moredevices comprises a neural interface probe provided in a massivelyparallel configuration.
 5. The system of claim 4, wherein the neuralinterface probe comprises a microwire bundle bonded onto a CMOS sensingarray.
 6. The system of claim 4, wherein the neural interface probecomprises an array of silicon microelectrode probes that are bonded ontoa CMOS sensing array such that each electrode site is routed to a uniquearray position.
 7. The system of claim 1, wherein the one or moredevices comprises an electroencephalography (EEG) device.
 8. The systemof claim 1, wherein the neural data is represented as one or more analogor digital signals representing activity recorded in the nervous system.9. The system of claim 1, where the neural data is stored frompreviously recorded activity in the nervous system of the living subjector other living subjects.
 10. The system of claim 1, wherein the sensingmodule comprises a plurality of sensors selected from the groupconsisting of vision sensors, audio sensors, touch sensors, locationsensors, inertial sensors, proximity sensors, heart rate monitors,temperature sensors, altitude sensors, attitude sensors, pressuresensors, humidity sensors, vibration sensors, chemical sensors, andelectromagnetic field sensors.
 11. The system of claim 10, wherein oneor more of the plurality of sensors in the sensing module are providedin a mobile device, or in a wearable device configured to be worn by theliving subject.
 12. The system of claim 5, wherein a plurality of neuralinterface probes are implanted in different areas of the brain of theliving subject that are associated with different sensory processing.13. The system of claim 12, wherein the different sensory processingincludes visual, auditory, tactile, taste, smell, position/movement,and/or interoception processing.
 14. The system of claim 13, wherein oneor more sensors are configured to collect real-world environmentalstimuli related to each of the different sensory processing.
 15. Thesystem of claim 14, wherein at least one neural interface probe isimplanted in an area of the living subject's brain associated withauditory processing, and wherein the sensing module comprises at leastone microphone configured to collect audio data in the vicinity of theliving subject.
 16. The system of claim 14, wherein at least one neuralinterface probe is implanted in an area of the living subject's brainassociated with visual processing, and wherein the sensing modulecomprises at least one camera configured to collect image data of theliving subject's surrounding.
 17. The system of claim 14, wherein atleast one neural interface probe is implanted in an area of the livingsubject's brain associated with spatial or location awareness, andwherein the sensing module comprises at least one global positioningsensor (GPS) sensor configured to collect positional data of the livingsubject.
 18. The system of claim 1, wherein the correspondence module isconfigured to determine correspondence between the neural data and thesensor data.
 19. The system of claim 1, wherein the correspondencemodule is configured to determine correspondence between different setsof neural data. 20.-22. (canceled)
 23. The system of claim 1, whereinthe correspondence module further comprises a decoder configured toreconstruct neural representations of sensory perceptions from theneural data. 24.-110. (canceled)